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Overview 

• 32 researchers (14 Ph.D.’s) 

• 25+ student interns yearly 

• 80% NASA work 

• 20% non-NASA work 

Research themes 

• Automated planetary mapping 

■ Base maps & terrain models 

■ Geospatial data systems 

• Exploration user interfaces 

■ Robot & science operations 

■ Accessible science data 

• Mobile robots 

■ Remotely operated & supervised 

■ Free-flyers, lake lander, & rovers 
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Robots 
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User Interfaces 









Visual Environment for Remote 
Virtual Exploration (VERVE) 
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Exploration Ground Data System (xGDS) 
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Remote Operation of Space Robots 



Part 1 

Operator on Ground 
Robot in Space 
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Part 2 

Operator in Space 
Robot on Ground 



Part 3 

Operator on Ground 
Robot on the Moon 
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Operator on Ground / Robot in Space 
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Robots for Human Exploration 


Motivation 

• Help maintain spacecraft 

• Enhance crew productivity 

• Perform work before, in support, 
and after humans 

In-Flight Maintenance (IFM) 



IN-FLIGHT MAINTENANCE TASKS 


• Must perform IFM to keep spacecraft 
in a safe, habitable configuration 

• Many IFM tasks are tedious, time- 
consuming, repetitive & routine 

• Many IFM tasks cannot be done 
using only fixed sensors / actuators 

Unmanned mission phases 

• Setup spacecraft prior to human 
arrival (e.g., Mars exploration) 

• Contingency situations 
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Inspect & monitor 

• Provide mobile camera views 

• Routine surveys and inventory 

• Check payload status / health 
Routine maintenance 

• Change air/water filters 

• Perform water draw/input on ECLSS 

• Payload adjustment & trouble shooting 
Contingency response 

• Assess environment after fire event 

• Assess & repair Leaks/MMOD damage 

• Power cycle/reboot electrical equipment 

• Actuate mechanisms (hatches, valves, etc.) 
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Space Station In-Flight Maintenance 


Extra-Vehicular Activity (EVA) 

• Not enough crew time to do everything 

(only 1-2 EVAs per year) 

• Crew must always carry out “Big 12” 
contingency EVA’s if needed 

■ Maintain electrical power system 

■ Maintain thermal control system 

• Prep & tear down: up to 3 hr per EVA 


Intra-Vehicular Activity (IVA) 

• Crew spends a lot of IVA time on 
maintenance (40+ hr/month) 


• Routine surveys require 12+ hr/month 

■ Air quality, lighting, sound level, 
video safety, etc. 


• Crew must always carry out 
contingency IVA surveys 



■ Find and repair leaks, etc. 
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Repetitive and Routine 


Camera positioning 

• Many cameras on the Space Station 

• Crew has to manually reposition 

cameras monitored by mission control 

• Camera are essential for many tasks 

■ Safety surveys 

■ Equipment and payload inspections 

■ Crew “over the shoulder” views 
during IVA activities 

Logistics 

• Crew must locate equipment and 
materials needed for IVA work 

■ Crew spends up to 1 hr per day 
manually searching for items 

■ 20,000 items in the inventory database 

• Automated logistics is a key NASA 
priority for future missions 
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Tasks 
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Smart SPHERES 





Space Telerobotics 
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Smart SPHERES 


Smartphone Upgrade 

• Delivered on STS-135 shuttle flight 

• Provides low-cost, off-the-shelf 
avionics upgrade for SPHERES 

• Activiated and initial check-out on 
November 1, 2011 

Key Points 

• Smartphone was the first commercial 
smartphone certified for use on the 
Space Station 

• Smartphone enables real-time, 
remote operation of SPHERES by 
crew and ground control 

• Smartphone provides modern CPU, 
Wi-Fi, and sensors (camera, 
magnetometer, etc) 
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Smart SPHERES Network Setup 
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Ground Control Test (12 December 2012) 


Space Station Interior Survey 

• Demonstrate free-flying video survey within (Kibo Laboratory module) 

• Smart SPHERES remotely operated from Mission Control (Houston) 

• Manual control (discrete commanding) and supervisory control 

(command sequences) 



OVHD 



x Space Telerobotics 
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STBD 



Space Station Interior Survey 



December 12, 2012 

Crew: Kevin Ford, Expedition 33 Commander 2x speed 
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Global Exploration Roadmap 



International Space Exploration 
Coordination Group 


Tele-Presence (p. 22) 

Tele-presence can be defined as tele-operation of a robotic 
asset on a planetary surface by a person who is relatively 
close to the planetary surface, perhaps orbiting in a space- 
craft or positioned at a suitable Lagrange point. Tele-presence 
is a capability which could significantly enhance the ability 
of humans and robots to explore together, where the specific 
exploration tasks would benefit from this capability. These 
tasks could be characterized by: 

• High-speed mobility 

• Short mission durations 

• Focused or dexterous tasks with short-time decision-making 

• Reduced autonomy or redundancy on the surface asset 

• Contingency modes/failure analysis through crew interaction 


x Space Telerobotics 
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Surface Telerobotics P 


Summary 

• Demo crew-control surface telerobotics 
(planetary rover) from ISS 

• Test human-robot conops for 

future exploration mission 

• Obtain baseline engineering data 

(robot, crew, data comm, task, etc) 

Implementation 

• Lunar libration mission simulation 

• Astronaut on Space Station 

• K10 rover in NASA Ames Roverscape 

ISS Testing (Expedition 36) 

17 June 2013 - C. Cassidy, survey 
26 July 2013 - L. Parmitano, deploy 
20 August 2013 - K. Nyberg, inspect 


Space Telerobotics 



• Human-robot mission sim: site survey, 
telescope deployment, and inspection 

• Telescope proxy: Kapton polyimide film roll 
(no antenna traces, electronics, or receiver) 

• 3.5 hr per crew session (“just in time” training, 
system checkout, ops, & debrief) 

• Robot ops: manual control (discrete commands) 
and supervisory control (task sequence) 
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INSPECT DEPLOY SURVEY 






“Fastnet” Lunar Libration Point Mission 





Orion MPCV at Earth-Moon L2 (EM-L2) 

• 60,000 km beyond lunar farside 

• Allows station keeping with minimal fuel 

• Crew remotely operates robot 

• Does not require human-rated lander 


Surface 
Mission Start 


Human-robot conops 

• Crew remotely operates surface robot 
from inside flight vehicle 

• Crew works in shirt-sleeve environment 

• Multiple robot control modes 


Earth/* 


Moon’s 


Depart Free- Return 
Trajectory 

V 


Orbit 
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Moon 


Surface ♦ 
Mission End 


Space Telerobotics 


19 


Credit: (Lockheed Martin / LUNAR) 




“Fastnet” Mission Simulation with ISS 


Planning 


Phase 1 


Phase 2 


Phase 3 | 


Pre-Mission 

Planning 



Ground teams 
plan out telescope 
deployment and 
initial rover 
traverses. 


Surveying 



Crew gathers 
information needed 
to finalize the 
telescope 
deployment plan. 


Telescope 

Deployment 



Crew monitors the 
rover as it deploys 
each arm of the 
telescope array. 


Telescope 

Inspection 



Crew inspects and 
documents the 
deployed telescope 
for possible 
damage. 





Crew Session 1 | 

Crew Session 2 | 

Crew Session 3 | 

Spring 2013 

17 June 2013 

26 July 2013 

20 August 2013 
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ISS Test Setup 




Interface 

Instrumentation & 
Evaluation Data 


Rover Plan 

(command sequence) 


3 kbit/sec (avg), 500 msec delay (max) 


“Live” Rover 
Sensor and 
Instrument 
Data 

(telemetry) 


K1 0 rover at NASA Ames 


500 msec delay (max) 



Space Telerobotics 


21 







Robot Interface (Supervisory Control) 


Task 

Sequence 
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Terrain hazards 
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Rover camera 
display 
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Robot Interface (Manual Control) 


Motion 

controls 


Camera 

controls 




Rover path 
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Terrain hazards 


Rover camera 
display 
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Surface Telerobotics 





Mountain View, California 


Roverscape 

Robotics Research, 
Development and Test Facility 

Intelligent Robotics Group 
NASA Ames Research Center 
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Assessment Approach 


Metrics 

• Mission Success: % task sequences: completed normally, ended abnormally 
or not attempted; % task sequences scheduled vs. unscheduled 

• Robot Utilization: % time robot spent on different types of tasks; comparison 
of actual to expected time on; did rover drive expected distance 

• Task Success: % task sequences per session and per task sequence: 
completed normally, ended abnormally or not attempted; % that ended 
abnormally vs. unscheduled task sequences 

• Contingencies: Mean Time To Intervene, Mean Time Between Interventions 

• Robot Performance: expected vs. actual execution time on tasks 

Data Collection 

”• Data Communication: direction (up/down), message type, total volume, etc. 

• Robot Telemetry: position, orientation, power, health, instrument state, etc. 

• User Interfaces: mode changes, data input, access to reference data, etc. 

_• Robot Operations: start, end, duration of planning, monitoring, and analysis 

• Crew Questionnaires: workload (Bedford Scale), situation awareness (SAGAT) 

M. Bualat, D. Schreckenghost, et al. (2014) “Results from testing crew-controlled surface 
telerobotics on the International Space Station”. Proc. of 12 th l-SAIRAS (Montreal, Canada) 
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Human-Robot Teaming 


Productivity Analysis 

• Productive Time (PT) = astronaut and robot performing tasks 
contributing to mission objectives 

• Overhead Time (OT) = astronaut and robot are waiting 

• %PT = percentage productive time 

• %OT = percentage overhead time 

• Work Efficiency Index (WEI) = Productive Time / Overhead Time 


Productivity 

Total Phase Time 

PT 

OT 

%PT 

%OT 

WEI 

Survey 

0:50:01 

0:34:58 

0:15:03 

69.90 

30.10 

2.32 

Deploy 

0:46:19 

0:28:00 

0:18:19 

60.45 

39.55 

1.53 


Highly productive 
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Operator on Ground / Robot on the Moon 


Resource Prospector Mission (2020) 



NASA led lunar rover mission 

• $200M (plus launch & lander) 

• NASA Class D / Category III project 

• Risk-tolerant, streamlined approach 


Partnerships 

• NASA to provide rover and payload 

• Detailed discussions and study 
with multiple partners for lander 


Status 

• Completed Mission Concept Review 
(MCR) in September 2013 

• Rover Engineering Test Unit (ETU) 
completed in August 2015 


NASA lander concept 
with rover and ISRU payload 


• Payload Engineering Units in test 

• Early 2020 launch date to the Moon 
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Resource Prospector Mission (2020) 


x Space Telerobotics 
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Rover Field Test (October 2014) 



• Prospecting. Mature prospecting ops concept for NIRVSS and NSS 
instruments in a lunar analog field test 

• Real-Time Operations. Improve support software by testing in a setting 
where the abundance / distribution of water is not known a priori 



• Science on Earth. Understand the emplacement and retention of water 
in the Mojave Desert by mapping water distribution / variability 

Space Telerobotics 
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Real-time Operations (NASA Ames) 



Space Telerobotics 
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Rover and Instruments 
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Rover Field Test (October 2014) 
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Current Work 






Questions? 



Intelligent Robotics Group 

Intelligent Systems Division 
NASA Ames Research Center 

irg.arc.nasa.gov 
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